Holographic superimposition of real world plenoptic opacity modulation through transparent waveguide arrays for light field, virtual and augmented reality

ABSTRACT

Disclosed are transparent energy relay waveguide systems for the superimposition of holographic opacity modulation states for holographic, light field, virtual, augmented and mixed reality applications. The light field system may comprise one or more energy waveguide relay systems with one or more energy modulation elements, each energy modulation element configured to modulate energy passing therethrough, whereby the energy passing therethrough may be directed according to 4D plenoptic functions or inverses thereof.

TECHNICAL FIELD

This disclosure generally relates to light field and 4D energymodulation systems, and more specifically, to holographic propagationthrough energy waveguide relay systems for superimposition of digitalopacity into a real-world coordinate system.

BACKGROUND

The dream of an interactive virtual world within a “holodeck” chamber aspopularized by Gene Roddenberry's Star Trek and originally envisioned byauthor Alexander Moszkowski in the early 1900s has been the inspirationfor science fiction and technological innovation for nearly a century.However, no compelling implementation of this experience exists outsideof literature, media, and the collective imagination of children andadults alike.

SUMMARY

Disclosed are transparent energy relay waveguide systems for thesuperimposition of holographic opacity modulation states forholographic, light field, virtual, augmented and mixed realityapplications.

In one embodiment, a transparent display system includes: a first energywaveguide relay system configured such that energy passing therethroughis directed according to a first 4D plenoptic function; a second energywaveguide relay system following the first energy waveguide relaysystem, the second energy waveguide relay system configured such thatenergy passing therethrough is directed according to a second 4Dplenoptic function, the second 4D plenoptic function inverse of thefirst 4D plenoptic function; and a first energy modulation elementdisposed in a first location in the first energy waveguide relay system,in a second location in the second energy waveguide relay system or in athird location in between the first energy waveguide relay system andthe second energy waveguide relay system, the first energy modulationelement configured to modulate energy passing therethrough.

In some embodiments, the first energy waveguide relay system includes afirst array of energy waveguides configured to direct energytherethrough along a plurality of energy propagation paths, where theenergy waveguides of the first array are located at different spatialcoordinates, and where each energy waveguide directs energy from therespective spatial coordinate to the plurality of energy propagationpaths along different directions according to the first 4D plenopticfunction.

In other embodiments, the second energy waveguide relay system includesa second array of energy waveguides configured to direct energytherethrough along a plurality of energy propagation paths, where theenergy waveguides of the second array are located at different spatialcoordinates, and each energy waveguide directs energy from therespective spatial coordinate to the plurality of energy propagationpaths along different directions according to the second 4D plenopticfunction.

In one embodiment, the system further includes a second energymodulation element located in one of the first, second, or thirdlocation, the second energy modulation element configured to modulateenergy passing therethrough. In one embodiment, the first and secondenergy modulation elements are located at the same location. In anotherembodiment, the first and second energy modulation elements are locatedat different locations.

In one embodiment, the system further includes third energy modulationelement located in one of the first, second, or third location, thethird energy modulation element configured to modulate energy passingtherethrough. In some embodiments, the third energy modulation elementsand at least one of the first and second energy modulation elements arelocated at the same location.

In one embodiment, the first, second, and third energy modulationelements are located at the same location. In another embodiment, thefirst, second, and third energy modulation elements are located atdifferent locations.

In some embodiments, each of the first, second and third energymodulation elements include LCD, LED, DLP, OLED, LCOS, quantum dot, orother suitable energy modulating elements. In another embodiment, atleast one of the first energy waveguide relay system and the secondenergy waveguide relay system is curved. In yet another embodiment, boththe first energy waveguide relay system and the second energy waveguiderelay system are curved.

In one embodiment, a transparent system includes: a first energywaveguide relay system configured such that energy passing therethroughis directed according to a first 4D plenoptic function; a second energywaveguide relay system following the first energy waveguide relaysystem, the second energy waveguide relay system configured such thatenergy passing therethrough is directed according to a second 4Dplenoptic function, the second 4D plenoptic function inverse of thefirst 4D plenoptic function; a first energy modulation element disposedin a first location in the first energy waveguide relay system, in asecond location in the second energy waveguide relay system or in athird location in between the first energy waveguide relay system andthe second energy waveguide relay system, the first energy modulationelement configured to modulate energy passing therethrough; and a secondenergy modulation element located in one of the first, second, or thirdlocation, the second energy modulation element configured to modulateenergy passing therethrough.

In another embodiment, the first energy waveguide relay system includesa first array of energy waveguides configured to direct energytherethrough along a plurality of energy propagation paths, where theenergy waveguides of the first array are located at different spatialcoordinates, and each energy waveguide directs energy from therespective spatial coordinate to the plurality of energy propagationpaths along different directions according to the first 4D plenopticfunction.

In another embodiment, the second energy waveguide relay system includesa second array of energy waveguides configured to direct energytherethrough along a plurality of energy propagation paths, where theenergy waveguides of the second array are located at different spatialcoordinates, and each energy waveguide directs energy from therespective spatial coordinate to the plurality of energy propagationpaths along different directions according to the second 4D plenopticfunction.

In one embodiment, the first and second energy modulation elements arelocated at the same location. In another embodiment, the first andsecond energy modulation elements are located at different locations.

In some embodiments, the system further includes a third energymodulation element located in one of the first, second, or thirdlocation, the third energy modulation element configured to modulateenergy passing therethrough. In one embodiment, the third energymodulation elements and at least one of the first and second energymodulation elements are located at the same location.

In some embodiments, the first, second, and third energy modulationelements are located at the same location. In other embodiments, thefirst, second, and third energy modulation elements are located atdifferent locations.

In some embodiments, each of the first, second and third energymodulation elements include LCD, LED, DLP, OLED, LCOS, quantum dot, orother suitable energy modulating elements. In another embodiment, atleast one of the first energy waveguide relay system and the secondenergy waveguide relay system is curved. In yet another embodiment, boththe first energy waveguide relay system and the second energy waveguiderelay system are curved.

In another embodiment, a transparent system includes: a first energywaveguide relay system configured such that energy passing therethroughis directed according to a first 4D plenoptic function; a second energywaveguide relay system following the first energy waveguide relaysystem, the second energy waveguide relay system configured such thatenergy passing therethrough is directed according to a second 4Dplenoptic function, the second 4D plenoptic function inverse of thefirst 4D plenoptic function; a first energy modulation element disposedin a first location in the first energy waveguide relay system, in asecond location in the second energy waveguide relay system or in athird location in between the first energy waveguide relay system andthe second energy waveguide relay system, the first energy modulationelement configured to modulate energy passing therethrough; a secondenergy modulation element located in one of the first, second, or thirdlocation, the second energy modulation element configured to modulateenergy passing therethrough; and a third energy modulation elementlocated in one of the first, second, or third location, the third energymodulation element configured to modulate energy passing therethrough.

In one embodiment, the first energy waveguide relay system includes afirst array of energy waveguides configured to direct energytherethrough along a plurality of energy propagation paths, where theenergy waveguides of the first array are located at different spatialcoordinates, and each energy waveguide directs energy from therespective spatial coordinate to the plurality of energy propagationpaths along different directions according to the first 4D plenopticfunction.

In another embodiment, the second energy waveguide relay system includesa second array of energy waveguides configured to direct energytherethrough along a plurality of energy propagation paths, where theenergy waveguides of the second array are located at different spatialcoordinates, and each energy waveguide directs energy from therespective spatial coordinate to the plurality of energy propagationpaths along different directions according to the second 4D plenopticfunction.

In one embodiment, the first and second energy modulation elements arelocated at the same location. In another embodiment, the first andsecond energy modulation elements are located at different locations. Insome embodiments, the third energy modulation elements and at least oneof the first and second energy modulation elements are located at thesame location. In other embodiments, the first, second, and third energymodulation elements are located at the same location. In yet some otherembodiments, the first, second, and third energy modulation elements arelocated at different locations.

In some embodiments, each of the first, second and third energymodulation elements include LCD, LED, DLP, OLED, LCOS, quantum dot, orother suitable energy modulating elements. In another embodiment, atleast one of the first energy waveguide relay system and the secondenergy waveguide relay system is curved. In yet another embodiment, boththe first energy waveguide relay system and the second energy waveguiderelay system are curved.

In one embodiment, a transparent display system includes: a first energywaveguide relay system configured such that energy passing therethroughis directed according to a first 4D plenoptic function; a second energywaveguide relay system following the first energy waveguide relaysystem, the second energy waveguide relay system configured such thatenergy passing therethrough is directed according to a second 4Dplenoptic function, the second 4D plenoptic function inverse of thefirst 4D plenoptic function; a first energy modulation element disposedin the first energy waveguide relay system; a second energy modulationelement disposed in between the first energy waveguide relay system andthe second energy waveguide relay system; and a third energy modulationelement disposed in the second energy waveguide relay system, where thefirst, second and third energy modulation elements are configured tomodulate energy passing therethrough.

In one embodiment, the first energy waveguide relay system includes afirst array of energy waveguides configured to direct energytherethrough along a plurality of energy propagation paths, where theenergy waveguides of the first array are located at different spatialcoordinates, and each energy waveguide directs energy from therespective spatial coordinate to the plurality of energy propagationpaths along different directions according to the first 4D plenopticfunction.

In another embodiment, the second energy waveguide relay system includesa second array of energy waveguides configured to direct energytherethrough along a plurality of energy propagation paths, where theenergy waveguides of the second array are located at different spatialcoordinates, and each energy waveguide directs energy from therespective spatial coordinate to the plurality of energy propagationpaths along different directions according to the second 4D plenopticfunction.

In some embodiments, each of the first, second and third energymodulation elements include LCD, LED, DLP, OLED, LCOS, quantum dot, orother suitable energy modulating elements. In another embodiment, thesystem further includes one or more additional energy modulationelements located in one of the first, second, or third location, the oneor more additional energy modulation elements configured to modulateenergy passing therethrough.

In one embodiment, the one or more additional energy modulation elementsand at least one of the first, second and third energy modulationelements are located at the same location. In another embodiment, theone or more additional energy modulation elements and at least one ofthe first, second and third energy modulation elements are located atdifferent locations. In one embodiment, at least one of the first energywaveguide relay system and the second energy waveguide relay system iscurved. In another embodiment, both the first energy waveguide relaysystem and the second energy waveguide relay system are curved.

In some embodiments, the first energy modulation element includes whiteopacity, the second energy modulation element includes an additionalopacity or color, and the third energy modulation element includes blackopacity.

In one example, the first and third energy modulation element are bothOLEDs and the second energy modulation element is an LCD such that topresent an opaque black color the first modulation element is configuredto an off state, the second modulation element is configured to an onstate, and the third modulation element can be configured to an on stateor an off state.

In another example, the first and third energy modulation element areboth OLEDs and the second energy modulation element is an LCD such thatto present a transparent black color the first modulation element isconfigured to an off state, the second modulation element is configuredto an off state, and the third modulation element is configured to anoff state.

In one example, the first and third energy modulation element are bothOLEDs and the second energy modulation element is an LCD such that topresent an opaque red color the first modulation element is configuredto a red only on state, the second modulation element can be configuredto an on state or an off state, and the third modulation element isconfigured to an off state.

In another example, the first and third energy modulation element areboth OLEDs and the second energy modulation element is an LCD such thatto present a transparent red color the first modulation element isconfigured to a red only state at a first percentage, the secondmodulation element is configured to an off state, and the thirdmodulation element is configured to a red only state at a secondpercentage, the second percentage different than the first percentage.

In one example, the first and third energy modulation element are bothOLEDs and the second energy modulation element is an LCD such that topresent an opaque grey color the first modulation element is configuredto an on state at a first percentage, the second modulation element isconfigured to an on state, and the third modulation element isconfigured to an off state.

In another example, the first and third energy modulation element areboth OLEDs and the second energy modulation element is an LCD such thatto present a transparent grey color the first modulation element isconfigured to an on state at a first percentage, the second modulationelement is configured to an off state, and the third modulation elementis configured to an on state at a second percentage, the secondpercentage different than the first percentage.

These and other advantages of the present disclosure will becomeapparent to those skilled in the art from the following detaileddescription and the appended claims.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic diagram illustrating design parameters for anenergy directing system;

FIG. 2 is a schematic diagram illustrating an energy waveguide relaysystem having an active device area with a mechanical envelope;

FIG. 3 is a schematic diagram illustrating an energy relay system;

FIG. 4 is a schematic diagram illustrating an embodiment of energy relayelements adhered together and fastened to a base structure;

FIG. 5A is a schematic diagram illustrating an example of a relayedimage through multi-core optical fibers;

FIG. 5B is a schematic diagram illustrating an example of a relayedimage through an optical relay that exhibits the properties of theTransverse Anderson Localization principle;

FIG. 6 is a schematic diagram showing rays propagated from an energysurface to a viewer;

FIG. 7 illustrates an orthogonal view of an optical relay systemconsisting of multiple energy waveguide pairs, in accordance with oneembodiment of the present disclosure;

FIG. 8 illustrates an orthogonal view of an increasing size of eachenergy waveguide pair to account for the scaling of the field of viewfrom the eye relative to the distance from the eye, in accordance withone embodiment of the present disclosure;

FIG. 9 illustrates an orthogonal view of energy waveguides that are bothscaled and cylindrically radiating about the rotational axis of the eye,in accordance with one embodiment of the present disclosure;

FIG. 10 illustrates an orthogonal view of a planar approach relativelocations of each of three potential displays in reference to energywaveguide pairs, in accordance with one embodiment of the presentdisclosure;

FIG. 11A illustrates an orthogonal view of a configurations to bond twodisplays together to increase manufacturing and calibration efficiency,in accordance with one embodiment of the present disclosure;

FIG. 11B illustrates an orthogonal view of a configurations to bond twodisplays together to increase manufacturing and calibration efficiency,in accordance with one embodiment of the present disclosure;

FIG. 12 illustrates an orthogonal view of an alternative to a volumetricopacity generating display that directly inverts the rays of light suchthat one or more elements from system may be removed, in accordance withone embodiment of the present disclosure; and

FIG. 13 illustrates an orthogonal view of ray propagation paths along acollection of individual elements within a waveguide relay system havingmultiple energy waveguide relay pairs, in accordance with one embodimentof the present disclosure.

DETAILED DESCRIPTION

An embodiment of a Holodeck (collectively called “Holodeck DesignParameters”) provide sufficient energy stimulus to fool the humansensory receptors into believing that received energy impulses within avirtual, social and interactive environment are real, providing: 1)binocular disparity without external accessories, head-mounted eyewear,or other peripherals; 2) accurate motion parallax, occlusion and opacitythroughout a viewing volume simultaneously for any number of viewers; 3)visual focus through synchronous convergence, accommodation and miosisof the eye for all perceived rays of light; and 4) converging energywave propagation of sufficient density and resolution to exceed thehuman sensory “resolution” for vision, hearing, touch, taste, smell,and/or balance.

Based upon conventional technology to date, we are decades, if notcenturies away from a technology capable of providing for all receptivefields in a compelling way as suggested by the Holodeck DesignParameters including the visual, auditory, somatosensory, gustatory,olfactory, and vestibular systems.

In this disclosure, the terms light field and holographic may be usedinterchangeably to define the energy propagation for stimulation of anysensory receptor response. While initial disclosures may refer toexamples of electromagnetic and mechanical energy propagation throughenergy surfaces for holographic imagery and volumetric haptics, allforms of sensory receptors are envisioned in this disclosure.Furthermore, the principles disclosed herein for energy propagationalong propagation paths may be applicable to both energy emission andenergy capture.

Many technologies exist today that are often unfortunately confused withholograms including lenticular printing, Pepper's Ghost, glasses-freestereoscopic displays, horizontal parallax displays, head-mounted VR andAR displays (HMD), and other such illusions generalized as“fauxlography.” These technologies may exhibit some of the desiredproperties of a true holographic display, however, lack the ability tostimulate the human visual sensory response in any way sufficient toaddress at least two of the four identified Holodeck Design Parameters.

These challenges have not been successfully implemented by conventionaltechnology to produce a seamless energy surface sufficient forholographic energy propagation. There are various approaches toimplementing volumetric and direction multiplexed light field displaysincluding parallax barriers, hogels, voxels, diffractive optics,multi-view projection, holographic diffusers, rotational mirrors,multilayered displays, time sequential displays, head mounted display,etc., however, conventional approaches may involve a compromise on imagequality, resolution, angular sampling density, size, cost, safety, framerate, etc., ultimately resulting in an unviable technology.

To achieve the Holodeck Design Parameters for the visual, auditory,somatosensory systems, the human acuity of each of the respectivesystems is studied and understood to propagate energy waves tosufficiently fool the human sensory receptors. The visual system iscapable of resolving to approximately 1 arc min, the auditory system maydistinguish the difference in placement as little as three degrees, andthe somatosensory system at the hands are capable of discerning pointsseparated by 2-12 mm. While there are various and conflicting ways tomeasure these acuities, these values are sufficient to understand thesystems and methods to stimulate perception of energy propagation.

Of the noted sensory receptors, the human visual system is by far themost sensitive given that even a single photon can induce sensation. Forthis reason, much of this introduction will focus on visual energy wavepropagation, and vastly lower resolution energy waveguide relay systemscoupled within a disclosed energy waveguide surface may convergeappropriate signals to induce holographic sensory perception. Unlessotherwise noted, all disclosures apply to all energy and sensorydomains.

When calculating for effective design parameters of the energypropagation for the visual system given a viewing volume and viewingdistance, a desired energy surface may be designed to include manygigapixels of effective energy location density. For wide viewingvolumes, or near field viewing, the design parameters of a desiredenergy surface may include hundreds of gigapixels or more of effectiveenergy location density. By comparison, a desired energy source may bedesigned to have 1 to 250 effective megapixels of energy locationdensity for ultrasonic propagation of volumetric haptics or an array of36 to 3,600 effective energy locations for acoustic propagation ofholographic sound depending on input environmental variables. What isimportant to note is that with a disclosed bi-directional energy surfacearchitecture, all components may be configured to form the appropriatestructures for any energy domain to enable holographic propagation.

However, the main challenge to enable the Holodeck today involvesavailable visual technologies and electromagnetic device limitations.Acoustic and ultrasonic devices are less challenging given the orders ofmagnitude difference in desired density based upon sensory acuity in therespective receptive field, although the complexity should not beunderestimated. While holographic emulsion exists with resolutionsexceeding the desired density to encode interference patterns in staticimagery, state-of-the-art display devices are limited by resolution,data throughput and manufacturing feasibility. To date, no singulardisplay device has been able to meaningfully produce a light fieldhaving near holographic resolution for visual acuity.

Production of a single silicon-based device capable of meeting thedesired resolution for a compelling light field display may notpractical and may involve extremely complex fabrication processes beyondthe current manufacturing capabilities. The limitation to tilingmultiple existing display devices together involves the seams and gapformed by the physical size of packaging, electronics, enclosure, opticsand a number of other challenges that inevitably result in an unviabletechnology from an imaging, cost and/or a size standpoint.

The embodiments disclosed herein may provide a real-world path tobuilding the Holodeck.

Example embodiments will now be described hereinafter with reference tothe accompanying drawings, which form a part hereof, and whichillustrate example embodiments which may be practiced. As used in thedisclosures and the appended claims, the terms “embodiment”, “exampleembodiment”, and “exemplary embodiment” do not necessarily refer to asingle embodiment, although they may, and various example embodimentsmay be readily combined and interchanged, without departing from thescope or spirit of example embodiments. Furthermore, the terminology asused herein is for the purpose of describing example embodiments onlyand is not intended to be limitations. In this respect, as used herein,the term “in” may include “in” and “on”, and the terms “a,” “an” and“the” may include singular and plural references. Furthermore, as usedherein, the term “by” may also mean “from”, depending on the context.Furthermore, as used herein, the term “if” may also mean “when” or“upon,” depending on the context. Furthermore, as used herein, the words“and/or” may refer to and encompass any and all possible combinations ofone or more of the associated listed items.

Holographic System Considerations Overview of Light Field EnergyPropagation Resolution

Light field and holographic display is the result of a plurality ofprojections where energy surface locations provide angular, color andintensity information propagated within a viewing volume. The disclosedenergy surface provides opportunities for additional information tocoexist and propagate through the same surface to induce other sensorysystem responses. Unlike a stereoscopic display, the viewed position ofthe converged energy propagation paths in space do not vary as theviewer moves around the viewing volume and any number of viewers maysimultaneously see propagated objects in real-world space as if it wastruly there. In some embodiments, the propagation of energy may belocated in the same energy propagation path but in opposite directions.For example, energy emission and energy capture along an energypropagation path are both possible in some embodiments of the presentdisclosed.

FIG. 1 is a schematic diagram illustrating variables relevant forstimulation of sensory receptor response. These variables may includesurface diagonal 101, surface width 102, surface height 103, adetermined target seating distance 118, the target seating field of viewfield of view from the center of the display 104, the number ofintermediate samples demonstrated here as samples between the eyes 105,the average adult inter-ocular separation 106, the average resolution ofthe human eye in arcmin 107, the horizontal field of view formed betweenthe target viewer location and the surface width 108, the vertical fieldof view formed between the target viewer location and the surface height109, the resultant horizontal waveguide element resolution, or totalnumber of elements, across the surface 110, the resultant verticalwaveguide element resolution, or total number of elements, across thesurface 111, the sample distance based upon the inter-ocular spacingbetween the eyes and the number of intermediate samples for angularprojection between the eyes 112, the angular sampling may be based uponthe sample distance and the target seating distance 113, the totalresolution Horizontal per waveguide element derived from the angularsampling desired 114, the total resolution Vertical per waveguideelement derived from the angular sampling desired 115, device Horizontalis the count of the determined number of discreet energy sources desired116, and device Vertical is the count of the determined number ofdiscreet energy sources desired 117.

A method to understand the desired minimum resolution may be based uponthe following criteria to ensure sufficient stimulation of visual (orother) sensory receptor response: surface size (e.g., 84″ diagonal),surface aspect ratio (e.g., 16:9), seating distance (e.g., 128″ from thedisplay), seating field of view (e.g., 120 degrees or +/−60 degreesabout the center of the display), desired intermediate samples at adistance (e.g., one additional propagation path between the eyes), theaverage inter-ocular separation of an adult (approximately 65 mm), andthe average resolution of the human eye (approximately 1 arcmin). Theseexample values should be considered placeholders depending on thespecific application design parameters.

Further, each of the values attributed to the visual sensory receptorsmay be replaced with other systems to determine desired propagation pathparameters. For other energy propagation embodiments, one may considerthe auditory system's angular sensitivity as low as three degrees, andthe somatosensory system's spatial resolution of the hands as small as2-12 mm.

While there are various and conflicting ways to measure these sensoryacuities, these values are sufficient to understand the systems andmethods to stimulate perception of virtual energy propagation. There aremany ways to consider the design resolution, and the below proposedmethodology combines pragmatic product considerations with thebiological resolving limits of the sensory systems. As will beappreciated by one of ordinary skill in the art, the following overviewis a simplification of any such system design, and should be consideredfor exemplary purposes only.

With the resolution limit of the sensory system understood, the totalenergy waveguide element density may be calculated such that thereceiving sensory system cannot discern a single energy waveguideelement from an adjacent element, given:

${{{Surface}{Aspect}{Ratio}} = \frac{{Width}(W)}{{Height}(H)}}{{{Surface}{Horizontal}{Size}} = {{Surface}{Diagonal}*\left( \frac{1}{\sqrt{\left( {1 + \left( \frac{H}{W} \right)^{2}} \right.}} \right)}}{{{Surface}{Vertical}{Size}} = {{Surface}{Diagonal}*\left( \frac{1}{\sqrt{\left( {1 + \left( \frac{W}{H} \right)^{2}} \right.}} \right)}}{{{Horizontal}{Field}{of}{View}} = {2*{{atan}\left( \frac{{Surface}{Horizontal}{Size}}{2*{Seating}{Distance}} \right)}}}{{{Vertical}{Field}{of}{View}} = {2*{{atan}\left( \frac{{Surface}{Vertical}{Size}}{2*{Seating}{Distance}} \right)}}}{{{Horizontal}{Element}{Resoluation}} = {{Horizontal}{FoV}*\frac{60}{{Eye}{Resolution}}}}{{{Vertical}{Element}{Resoluation}} = {{Vertical}{FoV}*\frac{60}{{Eye}{Resolution}}}}$

The above calculations result in approximately a 32×18° field of viewresulting in approximately 1920×1080 (rounded to nearest format) energywaveguide elements being desired. One may also constrain the variablessuch that the field of view is consistent for both (u, v) to provide amore regular spatial sampling of energy locations (e.g. pixel aspectratio). The angular sampling of the system assumes a defined targetviewing volume location and additional propagated energy paths betweentwo points at the optimized distance, given:

${{{Sample}{Distance}} = \frac{{Inter} - {Ocular}{Distance}}{\left( {{{Number}{of}{Desired}{Intermediate}{Samples}} + 1} \right)}}{{{Angular}{Sampling}} = {{atan}\left( \frac{{Sample}{Distance}}{{Seating}{Distance}} \right)}}$

In this case, the inter-ocular distance is leveraged to calculate thesample distance although any metric may be leveraged to account forappropriate number of samples at a given distance. With the abovevariables considered, approximately one ray per 0.57° may be desired andthe total system resolution per independent sensory system may bedetermined, given:

${{{Locations}{Per}{{Element}(N)}} = \frac{{Seating}{FoV}}{{Angular}{Sampling}}}{{{Total}{Resolution}H} = {N*{Horizontal}{Element}{Resolution}}}{{{Total}{Resolution}V} = {N*{Vertical}{Element}{Resolution}}}$

With the above scenario given the size of energy surface and the angularresolution addressed for the visual acuity system, the resultant energysurface may desirably include approximately 400 k×225 k pixels of energyresolution locations, or 90 gigapixels holographic propagation density.These variables provided are for exemplary purposes only and many othersensory and energy metrology considerations should be considered for theoptimization of holographic propagation of energy. In an additionalembodiment, 1 gigapixel of energy resolution locations may be desiredbased upon the input variables. In an additional embodiment, 1,000gigapixels of energy resolution locations may be desired based upon theinput variables.

Current Technology Limitations Active Area, Device Electronics,Packaging, and the Mechanical Envelope

FIG. 2 illustrates a device 200 having an active area 220 with a certainmechanical form factor. The device 200 may include drivers 230 andelectronics 240 for powering and interface to the active area 220, theactive area having a dimension as shown by the x and y arrows. Thisdevice 200 does not take into account the cabling and mechanicalstructures to drive, power and cool components, and the mechanicalfootprint may be further minimized by introducing a flex cable into thedevice 200. The minimum footprint for such a device 200 may also bereferred to as a mechanical envelope 210 having a dimension as shown bythe M:x and M:y arrows. This device 200 is for illustration purposesonly and custom electronics designs may further decrease the mechanicalenvelope overhead, but in almost all cases may not be the exact size ofthe active area of the device. In an embodiment, this device 200illustrates the dependency of electronics as it relates to active imagearea 220 for a micro OLED, DLP chip or LCD panel, or any othertechnology with the purpose of image illumination.

In some embodiments, it may also be possible to consider otherprojection technologies to aggregate multiple images onto a largeroverall display. However, this may come at the cost of greatercomplexity for throw distance, minimum focus, optical quality, uniformfield resolution, chromatic aberration, thermal properties, calibration,alignment, additional size or form factor. For most practicalapplications, hosting tens or hundreds of these projection sources 200may result in a design that is much larger with less reliability.

For exemplary purposes only, assuming energy devices with an energylocation density of 3840×2160 sites, one may determine the number ofindividual energy devices (e.g., device 100) desired for an energysurface, given:

${{{Devices}H} = \frac{{Total}{Resolution}H}{{Device}{Resolution}H}}{{{Devices}V} = \frac{{Total}{Resolution}V}{{Device}{Resolution}V}}$

Given the above resolution considerations, approximately 105×105 devicessimilar to those shown in FIG. 2 may be desired. It should be noted thatmany devices consist of various pixel structures that may or may not mapto a regular grid. In the event that there are additional sub-pixels orlocations within each full pixel, these may be exploited to generateadditional resolution or angular density. Additional signal processingmay be used to determine how to convert the light field into the correct(u,v) coordinates depending on the specified location of the pixelstructure(s) and can be an explicit characteristic of each device thatis known and calibrated. Further, other energy domains may involve adifferent handling of these ratios and device structures, and thoseskilled in the art will understand the direct intrinsic relationshipbetween each of the desired frequency domains. This will be shown anddiscussed in more detail in subsequent disclosure.

The resulting calculation may be used to understand how many of theseindividual devices may be desired to produce a full resolution energysurface. In this case, approximately 105×105 or approximately 11,080devices may be desired to achieve the visual acuity threshold. Thechallenge and novelty exists within the fabrication of a seamless energysurface from these available energy locations for sufficient sensoryholographic propagation.

Summary of Seamless Energy Surfaces Configurations and Designs forArrays of Energy Relays

In some embodiments, approaches are disclosed to address the challengeof generating high energy location density from an array of individualdevices without seams due to the limitation of mechanical structure forthe devices. In an embodiment, an energy propagating relay system mayallow for an increase the effective size of the active device area tomeet or exceed the mechanical dimensions to configure an array of relaysand form a singular seamless energy surface.

FIG. 3 illustrates an embodiment of such an energy relay system 300. Asshown, the relay system 300 may include a device 310 mounted to amechanical envelope 320, with an energy relay element 330 propagatingenergy from the device 310. The relay element 330 may be configured toprovide the ability to mitigate any gaps 340 that may be produced whenmultiple mechanical envelopes 320 of the device are placed into an arrayof multiple devices 310.

For example, if a device's active area 310 is 20 mm×10 mm and themechanical envelope 320 is 40 mm×20 mm, an energy relay element 330 maybe designed with a magnification of 2:1 to produce a tapered form thatis approximately 20 mm×10 mm on a minified end (arrow A) and 40 mm×20 mmon a magnified end (arrow B), providing the ability to align an array ofthese elements 330 together seamlessly without altering or collidingwith the mechanical envelope 320 of each device 310. Mechanically, therelay elements 330 may be bonded or fused together to align and polishensuring minimal seam gap 340 between devices 310. In one suchembodiment, it is possible to achieve a seam gap 340 smaller than thevisual acuity limit of the eye.

FIG. 4 illustrates an example of a base structure 400 having energyrelay elements 410 formed together and securely fastened to anadditional mechanical structure 430. The mechanical structure of theseamless energy surface 420 provides the ability to couple multipleenergy relay elements 410, 450 in series to the same base structurethrough bonding or other mechanical processes to mount relay elements410, 450. In some embodiments, each relay element 410 may be fused,bonded, adhered, pressure fit, aligned or otherwise attached together toform the resultant seamless energy surface 420. In some embodiments, adevice 480 may be mounted to the rear of the relay element 410 andaligned passively or actively to ensure appropriate energy locationalignment within the determined tolerance is maintained.

In an embodiment, the seamless energy surface comprises one or moreenergy locations and one or more energy relay element stacks comprise afirst and second side and each energy relay element stack is arranged toform a singular seamless display surface directing energy alongpropagation paths extending between one or more energy locations and theseamless display surface, and where the separation between the edges ofany two adjacent second sides of the terminal energy relay elements isless than the minimum perceptible contour as defined by the visualacuity of a human eye having better than 20/40 vision at a distancegreater than the width of the singular seamless display surface.

In an embodiment, each of the seamless energy surfaces comprise one ormore energy relay elements each with one or more structures forming afirst and second surface with a transverse and longitudinal orientation.The first relay surface has an area different than the second resultingin positive or negative magnification and configured with explicitsurface contours for both the first and second surfaces passing energythrough the second relay surface to substantially fill a +/−10 degreeangle with respect to the normal of the surface contour across theentire second relay surface.

In an embodiment, multiple energy domains may be configured within asingle, or between multiple energy relays to direct one or more sensoryholographic energy propagation paths including visual, acoustic, tactileor other energy domains.

In an embodiment, the seamless energy surface is configured with energyrelays that comprise two or more first sides for each second side toboth receive and emit one or more energy domains simultaneously toprovide bi-directional energy propagation throughout the system.

In an embodiment, the energy relays are provided as loose coherentelements.

Introduction to Component Engineered Structures Disclosed Advances inTransverse Anderson Localization Energy Relays

The properties of energy relays may be significantly optimized accordingto the principles disclosed herein for energy relay elements that induceTransverse Anderson Localization. Transverse Anderson Localization isthe propagation of a ray transported through a transversely disorderedbut longitudinally consistent material.

This implies that the effect of the materials that produce the AndersonLocalization phenomena may be less impacted by total internal reflectionthan by the randomization between multiple-scattering paths where waveinterference can completely limit the propagation in the transverseorientation while continuing in the longitudinal orientation.

Of significant additional benefit is the elimination of the cladding oftraditional multi-core optical fiber materials. The cladding is tofunctionally eliminate the scatter of energy between fibers, butsimultaneously act as a barrier to rays of energy thereby reducingtransmission by at least the core to clad ratio (e.g., a core to cladratio of 70:30 will transmit at best 70% of received energytransmission) and additionally forms a strong pixelated patterning inthe propagated energy.

FIG. 5A illustrates an end view of an example of one such non-AndersonLocalization energy relay 500, wherein an image is relayed throughmulti-core optical fibers where pixilation and fiber noise may beexhibited due to the intrinsic properties of the optical fibers. Withtraditional multi-mode and multi-core optical fibers, relayed images maybe intrinsically pixelated due to the properties of total internalreflection of the discrete array of cores where any cross-talk betweencores will reduce the modulation transfer function and increaseblurring. The resulting imagery produced with traditional multi-coreoptical fiber tends to have a residual fixed noise fiber pattern similarto those shown in FIG. 3 .

FIG. 5B, illustrates an example of the same relayed image 550 through anenergy relay comprising materials that exhibit the properties ofTransverse Anderson Localization, where the relayed pattern has agreater density grain structures as compared to the fixed fiber patternfrom FIG. 5A. In an embodiment, relays comprising randomized microscopiccomponent engineered structures induce Transverse Anderson Localizationand transport light more efficiently with higher propagation ofresolvable resolution than commercially available multi-mode glassoptical fibers.

There is significant advantage to the Transverse Anderson Localizationmaterial properties in terms of both cost and weight, where a similaroptical grade glass material, may cost and weigh upwards of 10 to100-fold more than the cost for the same material generated within anembodiment, wherein disclosed systems and methods comprise randomizedmicroscopic component engineered structures demonstrating significantopportunities to improve both cost and quality over other technologiesknown in the art.

In an embodiment, a relay element exhibiting Transverse AndersonLocalization may comprise a plurality of at least two differentcomponent engineered structures in each of three orthogonal planesarranged in a dimensional lattice and the plurality of structures formrandomized distributions of material wave propagation properties in atransverse plane within the dimensional lattice and channels of similarvalues of material wave propagation properties in a longitudinal planewithin the dimensional lattice, wherein localized energy wavespropagating through the energy relay have higher transport efficiency inthe longitudinal orientation versus the transverse orientation.

In an embodiment, multiple energy domains may be configured within asingle, or between multiple Transverse Anderson Localization energyrelays to direct one or more sensory holographic energy propagationpaths including visual, acoustic, tactile or other energy domains.

In an embodiment, the seamless energy surface is configured withTransverse Anderson Localization energy relays that comprise two or morefirst sides for each second side to both receive and emit one or moreenergy domains simultaneously to provide bi-directional energypropagation throughout the system.

In an embodiment, the Transverse Anderson Localization energy relays areconfigured as loose coherent or flexible energy relay elements.

Considerations for 4D Plenoptic Functions Selective Propagation ofEnergy Through Holographic Waveguide Arrays

As discussed above and herein throughout, a light field display systemgenerally includes an energy source (e.g., illumination source) and aseamless energy surface configured with sufficient energy locationdensity as articulated in the above discussion. A plurality of relayelements may be used to relay energy from the energy devices to theseamless energy surface. Once energy has been delivered to the seamlessenergy surface with the requisite energy location density, the energycan be propagated in accordance with a 4D plenoptic function through adisclosed energy waveguide system. As will be appreciated by one ofordinary skill in the art, a 4D plenoptic function is well known in theart and will not be elaborated further herein.

The energy waveguide system selectively propagates energy through aplurality of energy locations along the seamless energy surfacerepresenting the spatial coordinate of the 4D plenoptic function with astructure configured to alter an angular direction of the energy wavespassing through representing the angular component of the 4D plenopticfunction, wherein the energy waves propagated may converge in space inaccordance with a plurality of propagation paths directed by the 4Dplenoptic function.

Reference is now made to FIG. 6 illustrating an example of light fieldenergy surface in 4D image space in accordance with a 4D plenopticfunction. The figure shows ray traces of an energy surface 600 to aviewer 620 in describing how the rays of energy converge in space 630from various positions within the viewing volume. As shown, eachwaveguide element 610 defines four dimensions of information describingenergy propagation 640 through the energy surface 600. Two spatialdimensions (herein referred to as x and y) are the physical plurality ofenergy locations that can be viewed in image space, and the angularcomponents theta and phi (herein referred to as u and v), which isviewed in virtual space when projected through the energy waveguidearray. In general, and in accordance with a 4D plenoptic function, theplurality of waveguides (e.g., lenslets) are able to direct an energylocation from the x, y dimension to a unique location in virtual space,along a direction defined by the u, v angular component, in forming theholographic or light field system described herein.

However, one skilled in the art will understand that a significantchallenge to light field and holographic display technologies arisesfrom uncontrolled propagation of energy due designs that have notaccurately accounted for any of diffraction, scatter, diffusion, angulardirection, calibration, focus, collimation, curvature, uniformity,element cross-talk, as well as a multitude of other parameters thatcontribute to decreased effective resolution as well as an inability toaccurately converge energy with sufficient fidelity.

In an embodiment, an approach to selective energy propagation foraddressing challenges associated with holographic display may includeenergy inhibiting elements and substantially filling waveguide apertureswith near-collimated energy into an environment defined by a 4Dplenoptic function.

In an embodiment, an array of energy waveguides may define a pluralityof energy propagation paths for each waveguide element configured toextend through and substantially fill the waveguide element's effectiveaperture in unique directions defined by a prescribed 4D function to aplurality of energy locations along a seamless energy surface inhibitedby one or more elements positioned to limit propagation of each energylocation to only pass through a single waveguide element.

In an embodiment, multiple energy domains may be configured within asingle, or between multiple energy waveguides to direct one or moresensory holographic energy propagations including visual, acoustic,tactile or other energy domains.

In an embodiment, the energy waveguides and seamless energy surface areconfigured to both receive and emit one or more energy domains toprovide bi-directional energy propagation throughout the system.

In an embodiment, the energy waveguides are configured to propagatenon-linear or non-regular distributions of energy, includingnon-transmitting void regions, leveraging digitally encoded,diffractive, refractive, reflective, grin, holographic, Fresnel, or thelike waveguide configurations for any seamless energy surfaceorientation including wall, table, floor, ceiling, room, or othergeometry based environments. In an additional embodiment, an energywaveguide element may be configured to produce various geometries thatprovide any surface profile and/or tabletop viewing allowing users toview holographic imagery from all around the energy surface in a360-degree configuration.

In an embodiment, the energy waveguide array elements may be reflectivesurfaces and the arrangement of the elements may be hexagonal, square,irregular, semi-regular, curved, non-planar, spherical, cylindrical,tilted regular, tilted irregular, spatially varying and/or multilayered.

For any component within the seamless energy surface, waveguide, orrelay components may include, but not limited to, optical fiber,silicon, glass, polymer, optical relays, diffractive, holographic,refractive, or reflective elements, optical face plates, energycombiners, beam splitters, prisms, polarization elements, spatial lightmodulators, active pixels, liquid crystal cells, transparent displays,or any similar materials exhibiting Anderson localization or totalinternal reflection.

Realizing the Holodeck Aggregation of Bi-Directional Seamless EnergySurface Systems to Stimulate Human Sensory Receptors within HolographicEnvironments

It is possible to construct large-scale environments of seamless energysurface systems by tiling, fusing, bonding, attaching, and/or stitchingmultiple seamless energy surfaces together forming arbitrary sizes,shapes, contours or form-factors including entire rooms. Each energysurface system may comprise an assembly having a base structure, energysurface, relays, waveguide, devices, and electronics, collectivelyconfigured for bi-directional holographic energy propagation, emission,reflection, or sensing.

In an embodiment, an environment of tiled seamless energy waveguiderelay systems are aggregated to form large seamless planar or curvedwalls including installations comprising up to all surfaces in a givenenvironment, and configured as any combination of seamless,discontinuous planar, faceted, curved, cylindrical, spherical,geometric, or non-regular geometries.

In an embodiment, aggregated tiles of planar surfaces form wall-sizedsystems for theatrical or venue-based holographic entertainment. In anembodiment, aggregated tiles of planar surfaces cover a room with fourto six walls including both ceiling and floor for cave-based holographicinstallations. In an embodiment, aggregated tiles of curved surfacesproduce a cylindrical seamless environment for immersive holographicinstallations. In an embodiment, aggregated tiles of seamless sphericalsurfaces form a holographic dome for immersive Holodeck-basedexperiences.

In an embodiment, aggregated tiles of seamless curved energy waveguidesprovide mechanical edges following the precise pattern along theboundary of energy inhibiting elements within the energy waveguidestructure to bond, align, or fuse the adjacent tiled mechanical edges ofthe adjacent waveguide surfaces, resulting in a modular and seamlessenergy waveguide system.

In a further embodiment of an aggregated tiled environment, energy ispropagated bi-directionally for multiple simultaneous energy domains. Inan additional embodiment, the energy surface provides the ability toboth display and capture simultaneously from the same energy surfacewith waveguides designed such that light field data may be projected byan illumination source through the waveguide and simultaneously receivedthrough the same energy surface. In an additional embodiment, additionaldepth sensing and active scanning technologies may be leveraged to allowfor the interaction between the energy propagation and the viewer incorrect world coordinates. In an additional embodiment, the energysurface and waveguide are operable to emit, reflect or convergefrequencies to induce tactile sensation or volumetric haptic feedback.In some embodiments, any combination of bi-directional energypropagation and aggregated surfaces are possible.

In an embodiment, the system comprises an energy waveguide capable ofbi-directional emission and sensing of energy through the energy surfacewith one or more energy devices independently paired withtwo-or-more-path energy combiners to pair at least two energy devices tothe same portion of the seamless energy surface, or one or more energydevices are secured behind the energy surface, proximate to anadditional component secured to the base structure, or to a location infront and outside of the FOV of the waveguide for off-axis direct orreflective projection or sensing, and the resulting energy surfaceprovides for bi-directional transmission of energy allowing thewaveguide to converge energy, a first device to emit energy and a seconddevice to sense energy, and where the information is processed toperform computer vision related tasks including, but not limited to, 4Dplenoptic eye and retinal tracking or sensing of interference withinpropagated energy patterns, depth estimation, proximity, motiontracking, image, color, or sound formation, or other energy frequencyanalysis. In an additional embodiment, the tracked positions activelycalculate and modify positions of energy based upon the interferencebetween the bi-directional captured data and projection information.

In some embodiments, a plurality of combinations of three energy devicescomprising an ultrasonic sensor, a visible electromagnetic display, andan ultrasonic emitting device are configured together for each of threefirst relay surfaces propagating energy combined into a single secondenergy relay surface with each of the three first surfaces comprisingengineered properties specific to each device's energy domain, and twoengineered waveguide elements configured for ultrasonic andelectromagnetic energy respectively to provide the ability to direct andconverge each device's energy independently and substantially unaffectedby the other waveguide elements that are configured for a separateenergy domain.

In some embodiments, disclosed is a calibration procedure to enableefficient manufacturing to remove system artifacts and produce ageometric mapping of the resultant energy surface for use withencoding/decoding technologies as well as dedicated integrated systemsfor the conversion of data into calibrated information appropriate forenergy propagation based upon the calibrated configuration files.

In some embodiments, additional energy waveguides in series and one ormore energy devices may be integrated into a system to produce opaqueholographic pixels.

In some embodiments, additional waveguide elements may be integratedcomprising energy inhibiting elements, beam-splitters, prisms, activeparallax barriers or polarization technologies in order to providespatial and/or angular resolutions greater than the diameter of thewaveguide or for other super-resolution purposes.

In some embodiments, the disclosed energy waveguide relay system mayalso be configured as a wearable bi-directional device, such as virtualreality (VR) or augmented reality (AR). In other embodiments, the energywaveguide relay system may include adjustment optical element(s) thatcause the displayed or received energy to be focused proximate to adetermined plane in space for a viewer. In some embodiments, thewaveguide array may be incorporated to holographic head-mounted-display.In other embodiments, the system may include multiple optical paths toallow for the viewer to see both the energy waveguide relay system and areal-world environment (e.g., transparent holographic display). In theseinstances, the system may be presented as near field in addition toother methods.

In some embodiments, the transmission of data comprises encodingprocesses with selectable or variable compression ratios that receive anarbitrary dataset of information and metadata; analyze said dataset andreceive or assign material properties, vectors, surface IDs, new pixeldata forming a more sparse dataset, and wherein the received data maycomprise: 2D, stereoscopic, multi-view, metadata, light field,holographic, geometry, vectors or vectorized metadata, and anencoder/decoder may provide the ability to convert the data in real-timeor off-line comprising image processing for: 2D; 2D plus depth, metadataor other vectorized information; stereoscopic, stereoscopic plus depth,metadata or other vectorized information; multi-view; multi-view plusdepth, metadata or other vectorized information; holographic; or lightfield content; through depth estimation algorithms, with or withoutdepth metadata; and an inverse ray tracing methodology appropriatelymaps the resulting converted data produced by inverse ray tracing fromthe various 2D, stereoscopic, multi-view, volumetric, light field orholographic data into real world coordinates through a characterized 4Dplenoptic function. In these embodiments, the total data transmissiondesired may be multiple orders of magnitudes less transmittedinformation than the raw light field dataset.

Plenoptic Opacity Modulation Through Transparent Waveguide Arrays

Although much of this disclosure pertains to enabling a sensoryholographic experience, a disclosed intermediate step comprises theintegration between virtual and augmented reality devices tosignificantly limit data and processing requirements for thesimultaneous propagation and rasterization of a complete sensoryholographic dataset. With other contemporary VR and AR technologies, thelack of modulated opacity states, resolution, and field of viewdramatically limits the acuity of the sensory experience.

As an alternative, a novel transparent waveguide relay system capable ofholographic convergence of modulated opacity states superimposed overreal-world environments are disclosed. A direct view bidirectionalenergy waveguide relay design capable of transmission of real world datawith overlaid light field modulated attenuation of digital and realworld illumination comprises HMD, holographic, sensory energypropagation, as well as traditional display applications. The energywaveguide relay design comprises multiple energy waveguides and energymodulation elements where there is a first pair and a second pair thatmay be separated by two focal lengths. The two pairs may be placed twofocal lengths apart as well. The transparent energy waveguide pairs maybe placed in front of the eye to relay external energy uninhibited orattenuated by opacity modulation, or at a visible location at apredetermined distance and may include additional waveguide elements.The rays from the second pair furthest from the eye relays an invertedpropagation path and the introduction of an additional pair of energywaveguides at this distance rectifies these rays to the once againappropriate ray directionality. The 4D plenoptic function in combinationwith a plurality of energy modulating devices provides for thepropagation of holographic near field or far field digital informationover real world coordinates with true 4D opacity and other modulatedelectromagnetic or sensory energies.

FIG. 7 illustrates an orthogonal view of a waveguide relay system 70consisting of multiple energy waveguide pairs 72A, 72B, 72C, 72D, inaccordance with one embodiment of the present disclosure. As shown inFIG. 7 , each energy waveguide pair may be a focal length f apart andmay be viewed from the left, at the location of a viewer's eye 74, tothe right 76, or vice versa. In some embodiments, additional waveguideelements, fewer waveguide elements and various separations areenvisioned within the scope of this disclosure and the specifiedembodiments should not be considered limiting in any way.

In one embodiment, the waveguide relay system 70 may include: a firstenergy waveguide relay system 72A, 72B configured such that energypassing therethrough is directed according to a first 4D plenopticfunction; a second energy waveguide relay system 72C, 72D following thefirst energy waveguide relay system 72A, 72B, the second energywaveguide relay system 72C, 72D configured such that energy passingtherethrough is directed according to a second 4D plenoptic function,the second 4D plenoptic function inverse of the first 4D plenopticfunction

In one embodiment, the system 70 includes a first energy modulationelement (e.g., 102A, 102B, 102C as best shown in FIGS. 10 and 11 )disposed in a first location in the first energy waveguide relay system72A, 72B, in a second location in the second energy waveguide relaysystem 72C, 72D or in a third location in between the first energywaveguide relay system 72A, 72B and the second energy waveguide relaysystem 72C, 72D, the first energy modulation element configured tomodulate energy passing therethrough. This will be described in moredetail in subsequent figures and discussion.

In some embodiments, the first energy waveguide relay system 72A, 72Bincludes a first array of energy waveguides configured to direct energytherethrough along a plurality of energy propagation paths, where theenergy waveguides of the first array are located at different spatialcoordinates, and where each energy waveguide directs energy from therespective spatial coordinate to the plurality of energy propagationpaths along different directions according to the first 4D plenopticfunction.

In other embodiments, the second energy waveguide relay system 72C, 72Dincludes a second array of energy waveguides configured to direct energytherethrough along a plurality of energy propagation paths, where theenergy waveguides of the second array are located at different spatialcoordinates, and each energy waveguide directs energy from therespective spatial coordinate to the plurality of energy propagationpaths along different directions according to the second 4D plenopticfunction.

In one embodiment, the system 70 further includes a second energymodulation element (e.g., 102A, 102B, 102C as best shown in FIGS. 10 and11 ) located in one of the first, second, or third location, the secondenergy modulation element configured to modulate energy passingtherethrough. In one embodiment, the first and second energy modulationelements are located at the same location. In another embodiment, thefirst and second energy modulation elements are located at differentlocations. This, too, will be described in more detail in subsequentfigures and discussion.

In one embodiment, the system further includes third energy modulationelement (e.g., 102A, 102B, 102C as best shown in FIGS. 10 and 11 )located in one of the first, second, or third location, the third energymodulation element configured to modulate energy passing therethrough.In some embodiments, the third energy modulation elements and at leastone of the first and second energy modulation elements are located atthe same location. This, too, will be described in more detail insubsequent figures and discussion.

In one embodiment, the first, second, and third energy modulationelements (e.g., 102A, 102B, 102C as best shown in FIGS. 10 and 11 ) arelocated at the same location. In another embodiment, the first, second,and third energy modulation elements (e.g., 102A, 102B, 102C as bestshown in FIGS. 10 and 11 ) are located at different locations.

In some embodiments, each of the first, second and third energymodulation elements (e.g., 102A, 102B, 102C as best shown in FIGS. 10and 11 ) include LCD, LED, DLP, OLED, LCOS, quantum dot, or othersuitable energy modulating elements. In another embodiment, at least oneof the first energy waveguide relay system 72A, 72B and the secondenergy waveguide relay system 72C, 72D is curved. In yet anotherembodiment, both the first energy waveguide relay system 72A, 72B andthe second energy waveguide relay system 72C, 72D are curved. This, too,will be described in more detail in subsequent figures and discussion.

An additional embodiment of this fundamental approach provides tiltedwaveguides that optimize energy propagation quality as viewed by theeye. This may be implemented, but not limited to, regionally varyingfunction, gradient based tilting, additional energy inhibiting elements,diffractive properties, refraction, reflection, gradient index,holographic optics, or the like, and/or may be incorporated into anypotential designs above or below. A spatially varying energy waveguidedesign provides two or more regions with defined waveguide parameters tooptimize for a specified design for the eye. The gradient-based functioninvolves the generation of varying waveguide optimizations for eachelement.

FIG. 8 illustrates an orthogonal view of a waveguide relay system 80with increasing size of each energy waveguide pair 82A, 82B, 82C, 82D toaccount for the scaling of the field of view from the eye 84 relative tothe distance from the eye 84, in accordance with one embodiment of thepresent disclosure. This embodiment provides for greater waveguidingefficiency to provide propagation functions more normal to the apertureof the visual system thereby more efficiently targeting a field of viewof the eye 84 through each element within the system 86.

Like above, the waveguide relay system 80 may include: a first energywaveguide relay system 82A, 82B configured such that energy passingtherethrough is directed according to a first 4D plenoptic function; asecond energy waveguide relay system 82C, 82D following the first energywaveguide relay system 82A, 82B, the second energy waveguide relaysystem 82C, 82D configured such that energy passing therethrough isdirected according to a second 4D plenoptic function, the second 4Dplenoptic function inverse of the first 4D plenoptic function.

In one embodiment, a first energy modulation element (e.g., 102A, 102B,102C as best shown in FIGS. 10 and 11 ) may be disposed in a firstlocation in the first energy waveguide relay system 82A, 82B, in asecond location in the second energy waveguide relay system 82C, 82D orin a third location in between the first energy waveguide relay system82A, 82B and the second energy waveguide relay system 82C, 82D, thefirst energy modulation element configured to modulate energy passingtherethrough. This will be described in more detail in subsequentfigures and discussion.

In another embodiment, a second energy modulation element (e.g., 102A,102B, 102C as best shown in FIGS. 10 and 11 ) may be located in one ofthe first, second, or third location, the second energy modulationelement configured to modulate energy passing therethrough.

In another embodiment, the first energy waveguide relay system 82A, 82Bmay include a first array of energy waveguides configured to directenergy therethrough along a plurality of energy propagation paths, wherethe energy waveguides of the first array are located at differentspatial coordinates, and each energy waveguide directs energy from therespective spatial coordinate to the plurality of energy propagationpaths along different directions according to the first 4D plenopticfunction.

In yet another embodiment, the second energy waveguide relay system 82C,82D may include a second array of energy waveguides configured to directenergy therethrough along a plurality of energy propagation paths, wherethe energy waveguides of the second array are located at differentspatial coordinates, and each energy waveguide directs energy from therespective spatial coordinate to the plurality of energy propagationpaths along different directions according to the second 4D plenopticfunction.

In one embodiment, the first and second energy modulation elements(e.g., 102A, 102B, 102C as best shown in FIGS. 10 and 11 ) are locatedat the same location. In another embodiment, the first and second energymodulation elements are located at different locations.

In some embodiments, the system 80 may further include a third energymodulation element (e.g., 102A, 102B, 102C as best shown in FIGS. 10 and11 ) located in one of the first, second, or third location, the thirdenergy modulation element configured to modulate energy passingtherethrough. In one embodiment, the third energy modulation elementsand at least one of the first and second energy modulation elements arelocated at the same location. In some embodiments, the first, second,and third energy modulation elements are located at the same location.In other embodiments, the first, second, and third energy modulationelements are located at different locations. In some embodiments, eachof the first, second and third energy modulation elements includes LCD,LED, DLP, OLED, LCOS, quantum dot, or other energy modulation elements.In another embodiment, at least one of the first energy waveguide relaysystem and the second energy waveguide relay system is curved. In yetanother embodiment, both the first energy waveguide relay system and thesecond energy waveguide relay system are curved. These embodiments willbe described in more detail in subsequent figures and discussion.

FIG. 9 illustrates an orthogonal view of a waveguide relay system 90 ofenergy waveguides 92A, 92B, 92C, 92D that are both scaled andcylindrically radiating about the rotational axis of the eye, inaccordance with one embodiment of the present disclosure. In thisembodiment, the energy waveguides 92A, 928, 92C, 92D are curved toaccount for the rotational axis of a viewer's eye 94. This may beimplemented as non-scaled or as a scaled and curved design. This curveddesign may be implemented as a horizontally or vertically cylindricalshape or may be formed as a concave or spherical shape. Use of thisapproach may increase the perceived clarity of the rays presentedthrough the relay system 90 to the eye 94. It should also be noted thata cylindrical or spherical approach provides greater efficiency of thenumber of angular samples per waveguide as the normal of each waveguideare more correctly oriented towards the entrance pupil of the eye 94,thereby more efficiently mimicking natural energy propagation ratherthan planar functions that propagate rays off axis for the periphery ofthe field of view of the eye 94.

As depicted in FIG. 9 , at least one of the first energy waveguide relaysystem 92A, 92B and the second energy waveguide relay system 92C, 92Dmay be curved. Although both energy waveguide relay systems 92 are shownto be curved, it will be appreciated by one skilled in the art that inone embodiment the first energy waveguide relay system 92A, 92B may becurved and the second energy waveguide relay system 92C, 92D may beplanar. In an alternative embodiment, the first energy waveguide relaysystem 92A, 92B may be planar while the second energy waveguide relaysystem 92C, 92D may be curved. The curving of the energy waveguide relaysystems 92 may similarly be applied to all the systems (e.g., FIGS. 7, 8and 10-12 ) disclosed herein.

Like above, in one embodiment, the system 90 may include: a first energywaveguide relay system 92A, 92B configured such that energy passingtherethrough is directed according to a first 4D plenoptic function; asecond energy waveguide relay system 92C, 92D following the first energywaveguide relay system 92A, 92B, the second energy waveguide relaysystem 92C, 92D configured such that energy passing therethrough isdirected according to a second 4D plenoptic function, the second 4Dplenoptic function inverse of the first 4D plenoptic function.

In another embodiment, a first energy modulation element (e.g., 102A,102B, 102C as best shown in FIGS. 10 and 11 ) may be disposed in a firstlocation in the first energy waveguide relay system 92A, 92B, in asecond location in the second energy waveguide relay system 92C, 92D orin a third location in between the first energy waveguide relay system92A, 92B and the second energy waveguide relay system 92C, 92D, thefirst energy modulation element configured to modulate energy passingtherethrough. This will be described in more detail in subsequentfigures and discussion.

In yet another embodiment, a second energy modulation element (e.g.,102A, 102B, 102C as best shown in FIGS. 10 and 11 ) may be located inone of the first, second, or third location, the second energymodulation element configured to modulate energy passing therethrough,and a third energy modulation element (e.g., 102A, 1028, 102C as bestshown in FIGS. 10 and 11 ) may be located in one of the first, second,or third location, the third energy modulation element configured tomodulate energy passing therethrough. This will be described in moredetail in subsequent figures and discussion.

In one embodiment, the first energy waveguide relay system 92A, 92Bincludes a first array of energy waveguides configured to direct energytherethrough along a plurality of energy propagation paths, where theenergy waveguides of the first array are located at different spatialcoordinates, and each energy waveguide directs energy from therespective spatial coordinate to the plurality of energy propagationpaths along different directions according to the first 4D plenopticfunction.

In another embodiment, the second energy waveguide relay system 92C, 92Dincludes a second array of energy waveguides configured to direct energytherethrough along a plurality of energy propagation paths, where theenergy waveguides of the second array are located at different spatialcoordinates, and each energy waveguide directs energy from therespective spatial coordinate to the plurality of energy propagationpaths along different directions according to the second 4D plenopticfunction.

In one embodiment, the first and second energy modulation elements(e.g., 102A, 102B, 102C as best shown in FIGS. 10 and 11 ) may belocated at the same location. In another embodiment, the first andsecond energy modulation elements may be located at different locations.In some embodiments, the third energy modulation elements and at leastone of the first and second energy modulation elements may be located atthe same location. In other embodiments, the first, second, and thirdenergy modulation elements may be located at the same location. In yetsome other embodiments, the first, second, and third energy modulationelements may be located at different locations. In some embodiments,each of the first, second and third energy modulation elements includesLCD, LED, DLP, OLED, LCOS, quantum dot, or other suitable energymodulating elements. These embodiments will be described in more detailin subsequent figures and discussion.

FIG. 10 illustrates an orthogonal view of a waveguide relay system 100showing the relative locations of each of three potential energymodulation devices 102A, 102B, 102C in relation to energy waveguides104A, 104B, 104C, 104D, in accordance with one embodiment of the presentdisclosure. In one embodiment, each pair of energy waveguides (e.g.,104A and 104B or 104C and 104D) may be configured to form an energywaveguide relay system.

In order to overlay a virtual light field onto the real world requiresthe integration of a plurality of energy propagation paths as viewedthrough the above-mentioned energy waveguide relay configurations. Relayof energy waveguides with three energy devices (transparent OLED 102A,transparent LCD 102B, and transparent OLED 102C, provided for exemplarypurposes only), enables true convergence of holographic opacity throughthe transmissive light field energy directing relay. In someembodiments, the explicit reference to a specific type of displaytechnology is for exemplary purposes only and not intended to limit thedisclosure in any way. A pairing of the three specially mentioned (butnot limited to) energy modulation devices 102A, 102B, 102C placed at orabout the focal length in between each of the pairs of energy waveguides104A, 104B, 104C, 104D, plus at the focal length f from a viewer's eye106 between the pairs 104A, 104B, 104C, 104D, the ability for thedigital superimposition of variable light field opacity and transmissivestates for any range of energies within a specified energy domain.

In an embodiment, a first energy modulation device may comprisestructures for volumetric spectral modulation of opaque energypropagations in an active state. Spectral modulation may additionally bepropagated through combination of other energy modulation device pairs102A, 102C. The specific modulation devices 102A, 102C may provideprecise control over propagation of saturation and transparency. For thepurposes of this filing, the addition of all three energy modulationdevices 102A, 102B, 102C may or may not be used—only one energymodulation device is needed for energy propagation. Designs may beimplemented in any combination, specifically referring to previousembodiments comprising planar waveguiding systems, curved, scaled,variable approaches as disclosed depending on the specific application.

As depicted in FIG. 10 , the system 100 may comprise a first energywaveguide relay system 104A, 104B configured such that energy passingtherethrough is directed according to a first 4D plenoptic function anda second energy waveguide relay system 104C, 104D following the firstenergy waveguide relay system 104A, 102A, 104B, the second energywaveguide relay system 104C, 102C, 104D configured such that energypassing therethrough is directed according to a second 4D plenopticfunction, the second 4D plenoptic function inverse of the first 4Dplenoptic function.

As depicted in FIG. 10 , a first modulation element 102A may be an LCD,LED, DLP, OLED, LCOS, quantum dot, or other suitable energy modulatingelements. The system may include a second modulation element 102B and athird modulation element 102C similar to that of the first modulationelement 102A.

In one embodiment, a first energy modulation element 102A may bedisposed in the first energy waveguide relay system 104A, 104B while asecond energy modulation element 102B may be disposed in between thefirst energy waveguide relay system 104A, 104B and the second energywaveguide relay system 104C, 104D. A third energy modulation element102C may be disposed in the second energy waveguide relay system 104C,104D. In this embodiment, each of the first, second and third energymodulation elements 102A, 102B, 102C may be configured to modulateenergy passing therethrough.

Although the first modulation element 102A is shown to be in the firstenergy waveguide relay system 104A, 104B, the second modulation element102B is shown to be in between the first energy waveguide relay system104A, 104B and the second energy waveguide relay system 104C, 104D, andthe third modulation element 102C is shown to be in the second energywaveguide relay system 104C, 104D, it will be understood that themodulation elements 102A, 102B, 102C can be located anywhere throughoutthe system 100. For example, all three modulation elements 102 may belocated in the first energy waveguide relay system 104A, 104B. Inanother embodiment, all three modulation elements 102 may be located inthe second energy 104C, 104D. In yet another embodiment, all threemodulation elements 102 may be located in between the first energywaveguide relay system 104A, 104B and the second energy waveguide relaysystem 104C, 104D. It will be understood by one of ordinary skill in theart that various combinations and permutations may be utilized. In someembodiments, the system 100 may further include four or five or as manymodulation elements 102 depending on the application.

Like that of FIG. 9 , the first energy waveguide relay system 104A, 104Bmay be curved in one embodiment. In another embodiment, the secondenergy waveguide relay system 104C, 104D may be curved. In yet anotherembodiment, both the first energy waveguide relay system 104A, 104B andthe second energy waveguide relay system 104C, 104D may be curved. Itwill be appreciated by one skilled in the art that although two energywaveguide relay systems 104 are shown, there can be three or four ormany more energy waveguide relay systems 104 as necessary depending onthe application. Furthermore, one or two or any number of these energywaveguide relay systems 104 may or may not be curved.

In one embodiment, the transparent display system 100 includes: a firstenergy waveguide relay system 104A, 104B configured such that energypassing therethrough is directed according to a first 4D plenopticfunction; a second energy waveguide relay system 104C, 104D followingthe first energy waveguide relay system 104A, 104B, the second energywaveguide relay system 104C, 104D configured such that energy passingtherethrough is directed according to a second 4D plenoptic function,the second 4D plenoptic function inverse of the first 4D plenopticfunction. In this embodiment, a first energy modulation element 102A isdisposed in the first energy waveguide relay system 104A, 104B; a secondenergy modulation element 102B is disposed in between the first energywaveguide relay system 104A, 104B and the second energy waveguide relaysystem 104C, 104D; and a third energy modulation element 102C isdisposed in the second energy waveguide relay system 104C, 104D, wherethe first, second and third energy modulation elements 102A, 102B, 102Care configured to modulate energy passing therethrough.

In one embodiment, the first energy waveguide relay system 104A, 104Bmay include a first array of energy waveguides configured to directenergy therethrough along a plurality of energy propagation paths,wherein the energy waveguides of the first array are located atdifferent spatial coordinates, and each energy waveguide directs energyfrom the respective spatial coordinate to the plurality of energypropagation paths along different directions according to the first 4Dplenoptic function.

In another embodiment, the second energy waveguide relay system 104C,104D may include a second array of energy waveguides configured todirect energy therethrough along a plurality of energy propagationpaths, wherein the energy waveguides of the second array are located atdifferent spatial coordinates, and each energy waveguide directs energyfrom the respective spatial coordinate to the plurality of energypropagation paths along different directions according to the second 4Dplenoptic function.

In some embodiments, the system 100 may further include one or moreadditional energy modulation elements (not shown) located in one of thefirst, second, or third locations, the one or more additional energymodulation elements configured to modulate energy passing therethrough.The first location may be within the first energy waveguide relay system104A, 104B, the second location may be between the first energywaveguide relay system 104A, 104B and the second energy waveguide relaysystem 104C, 104D, and the third location may be within the secondenergy waveguide relay system 104C, 104D. In some embodiments, the oneor more additional energy modulation elements (not shown) and at leastone of the first, second and third energy modulation elements 102 may belocated at the same location. In other embodiments, the one or moreadditional energy modulation elements (not shown) and at least one ofthe first, second and third energy modulation elements 102 may belocated at different locations.

In an embodiment, three energy modulation elements 102 cumulativelypropagate electromagnetic energies, comprising modulated amalgams forcolor, transparency, intensity, opacity and a plurality of otherholographic conditions. Table 1 is presented for reference of a fewvariable opacity and transmissive states for a plurality of spectralvalues and the commensurate exemplary value each of the three energymodulation elements would exhibit.

TABLE 1 Second First Modulation Modulation Third Modulation Element(102A) Element (102B) Element (102C) [OLED] [LCD] [OLED] Opaque BlackOff On Off or On Transparent Black Off Off Off Opaque Red Red Only On Onor Off Off Transparent Red Red Only X % Off Red Only Y % Opaque Grey OnX % On Off Transparent Grey On X % Off On Y %

Table 1 illustrates a matrix of potential color values for each of therespective modulation elements in the relay system to produce volumetricrays that include all color, intensity, transparency and opacity in fulllight field projection.

Values that indicate on or off a referring to the potential for eitherconfiguration depending on the efficiency of the modulation system toabsorb electromagnetic radiation in a configuration. For values thatindicate X or Y percent, this is due to the need to determine theefficiency of energy modulation device 102A aggregated through energymodulation device 102C to produce the effective and desired holographicopacity and spectral energy propagation. Y may be off or may be somepercent of the total transmission desired.

In some embodiments, the modulation of the energy waveguide relaysystems comprise all sensory energy domains, and wherein the energymodulation devices may comprise other forms of sensory energy devices toinclude visual, auditory, gustatory, olfactory, somatosensory or othersensory systems as disclosed. Further, the scope of this systemcomprises bidirectional capabilities between and through the energypropagating waveguide relay system to be leveraged for additionalembodiments comprising any such disclosures relating to the directing ofholographic or sensory energy. In an embodiment, the energy relay systemis designed as a bidirectional large format energy directing devicehaving both modulation and other energy devices to enable transparentand holographic superimposition of sensory energies through thewaveguiding relay system, and wherein the systems may appear invisiblethrough the transparent system, and may comprise energy sensors or anycombination of sensory energy propagation, holodeck parameters,interactivity or otherwise leveraged for other applications notdisclosed explicitly within this application.

It should be noted that this approach provides the ability to truly“paint” black or opaque values in volumetric space wherein the pluralityof modulation devices and waveguiding systems induces convergence ofenergy propagations paths such that the projected coordinate in spacecomprise the properties of real world objects. In an embodiment,interactivity, volumetric masking or other novel applications may beleveraged outside of the disclosed embodiments. This novel approach torecreate real world reflectance points from a near field, augmented,virtual or other head mounted display, is a significant leap inimmersive qualities and provides a greater sense of realism not possiblewith other approaches.

As shown in FIG. 10 and disclosed in Table 1, a first energy modulationelement may include white opacity, and a second energy modulationelement may include an additional opacity or color, and the third energymodulation element may include black opacity. In an embodiment, firstand third energy modulation elements may both be OLEDs 102A, 102C andthe second energy modulation element may be an LCD 102B such that topresent an opaque black color the first modulation element is configuredto an off state, the second modulation element is configured to an onstate, and the third modulation element can be configured to an on stateor an off state.

FIG. 13 illustrates a single energy waveguide relay element system 100,wherein a single effective collection of four waveguide elementfunctions comprising waveguide relay element pair 104A, 104B and 104C,104D provide for efficient energy propagation. Within the subset ofpropagated energy waves, 130A, 130B, 130C, the functions prescribed bythe energy waveguides when guided by a 4D function, allows for theaccurate convergence for given ray bundles propagating along the system,wherein, the knowledge of energy domain, frequency, or otherenvironmental parameters provides for both the projection and sensing ofenergy in accordance with a 4D function.

In reference to Table 1 and in consideration of FIG. 13 , one skilled inthe art will appreciate the method to converge modulated states through102A, 102B, 102C thereby intersecting propagation paths 130A, 130B, 130Cwherein the energy entering the system adjacent to waveguide element104D at location 131 may propagate bidirectionally through the system.In the event that energy propagates through the system from location 131to a viewed position 106, the energy waves propagating throughout thewaveguide functions 104A, 104B, 104C, 104D and energy modulating devices102A, 102B, 102C, exit with a direction that is substantially similarbetween entry paths 131 and exit paths 132. For viewed rays andsuperimposed information generated by energy modulating elements 102A,102B, and 102C from a viewed location 106 and propagating along 132, thebidirectional propagation of energy remains substantially similarthroughout the energy waveguide relay system though location 131.

In this fashion, it is possible to propagate the information through thetransparent waveguide relay system wherein the viewer 106 may receiveenergy 132 from 131 substantially unmodified from the original energywaves when modulation elements 102A, 102B, and 102C in the absence ofmodulation. However, upon activation of the energy modulation devicesguided by 4D functions provides the ability to propagate converging 4Dplenoptic opacity states, such that the apparent differential between atruly opaque object and a superimposed virtual object may beindistinguishable with sufficient calibration and maturation of thetechnology.

In an embodiment, first and third energy modulation element are bothOLEDs 102A, 102C and the second energy modulation element may be an LCD102B such that to present a transparent black color the first modulationelement is configured to an off state, the second modulation element isconfigured to an off state, and the third modulation element isconfigured to an off state.

In an embodiment, first and third energy modulation elements are bothOLEDs 102A, 102C and the second energy modulation element may be an LCD102B such that to present an opaque red color the first modulationelement is configured to a red only on state, the second modulationelement can be configured to an on state or an off state, and the thirdmodulation element is configured to an off state.

In an embodiment, first and third energy modulation elements are bothOLEDs 102A, 102C and the second energy modulation element may be an LCD102B such that to present a transparent red color the first modulationelement is configured to a red only state at a first percentage, thesecond modulation element is configured to an off state, and the thirdmodulation element is configured to a red only state at a secondpercentage, the second percentage different than the first percentage.

In an embodiment, first and third energy modulation elements are bothOLEDs 102A, 102C and the second energy modulation element may be an LCD102B such that to present an opaque grey color the first modulationelement is configured to an on state at a first percentage, the secondmodulation element is configured to an on state, and the thirdmodulation element is configured to an off state.

In an embodiment, first and third energy modulation elements are bothOLEDs 102A, 102C and the second energy modulation element may be an LCD1028 such that to present a transparent grey color the first modulationelement is configured to an on state at a first percentage, the secondmodulation element is configured to an off state, and the thirdmodulation element is configured to an on state at a second percentage,the second percentage different than the first percentage.

In some embodiments, the present disclosures may be implemented for anysize type explicitly to include displays that are not head mounted inany way. In this fashion, it is possible to “paint” transparency valueswith any form of light field, including visual or other sensory displaysystems, leveraging the method and systems presented herein.

In an embodiment for manufacturing efficiency, an additional embodimentproposes to bond together either energy modulation element 102A and 102Eor 102C and 102B. This helps increase the efficiency of calibration andsolve a number of mechanical alignment challenges.

FIG. 11A illustrates an orthogonal view of a configuration 110 to bondtwo energy modulation elements 102A, 102B together to increasemanufacturing and calibration efficiency, in accordance with oneembodiment of the present disclosure. FIG. 11B illustrates an orthogonalview of a configuration 116 to bond two energy modulation elements 102B,102C together to increase manufacturing and calibration efficiency, inaccordance with one embodiment of the present disclosure.

As depicted in FIGS. 11A and 11B, in an embodiment, the system 110 (FIG.11A) or 116 (FIG. 11B) may include a first energy waveguide relay system112A, 112B configured such that energy passing therethrough is directedaccording to a first 4D plenoptic function and a second energy waveguiderelay system 112C, 112D following the first energy waveguide relaysystem 112A, 112B, the second energy waveguide relay system 112C, 112Dconfigured such that energy passing therethrough is directed accordingto a second 4D plenoptic function, the second 4D plenoptic functioninverse of the first 4D plenoptic function.

In one embodiment, the system 110 may include an OLED modulation element102A, a LCD modulation element 102B, and another OLED modulation element102C, and any combinations thereof. As shown in FIG. 11A, the OLEDmodulation element 102A and the LCD modulation element 102B may belocated in the first energy waveguide relay system 112A, 112E while theOLED modulation element 102C may be located in the second energywaveguide relay system 112C, 112D. Conversely, as shown in FIG. 11B, theOLED modulation element 102A may be located in the first energywaveguide relay system 112A, 112B while the LCD modulation element 102Band the OLED modulation element 102C are located in the second energywaveguide relay system 112C, 112D. It will be appreciated by one skilledin the art that the modulation elements 102 maybe randomly distributedthroughout the energy waveguide relay systems 112.

In an embodiment, a first energy modulation element may be disposed in afirst location in the first energy waveguide relay system, in a secondlocation in the second energy waveguide relay system or in a thirdlocation in between the first energy waveguide relay system and thesecond energy waveguide relay system, with the first energy modulationelement configured to modulate energy passing therethrough. In anotherembodiment, a second energy modulation element may be located in one ofthe first, second, or third locations, with the second energy modulationelement configured to modulate energy passing therethrough. In yetanother embodiment, a third energy modulation element may be located inone of the first, second, or third locations, with the second energymodulation element configured to modulate energy passing therethrough.In one example, the third energy modulation element may be located atthe same location of at least one of the first and second energymodulation elements. In another example, the third energy modulationelement may be located at different locations from the at least one ofthe first and second energy modulation elements.

In one embodiment, the first energy waveguide relay system 112A, 112Bmay include a first array of energy waveguides configured to directenergy therethrough along a plurality of energy propagation paths,wherein the energy waveguides of the first array are located atdifferent spatial coordinates, and each energy waveguide directs energyfrom the respective spatial coordinate to the plurality of energypropagation paths along different directions according to the first 4Dplenoptic function. The second energy waveguide relay system may includea second array of energy waveguides configured to direct energytherethrough along a plurality of energy propagation paths, wherein theenergy waveguides of the second array are located at different spatialcoordinates, and each energy waveguide directs energy from therespective spatial coordinate to the plurality of energy propagationpaths along different directions according to the second 4D plenopticfunction.

The second energy modulation element may be located in one of the first,second, or third locations, with the second energy modulation elementconfigured to modulate energy passing therethrough. In an embodiment,the first and second energy modulation elements are located at the samelocation, but the first and second energy modulation elements may alsobe located at different locations.

In some embodiments, a third energy modulation element may be located inone of the first, second, or third locations, the third energymodulation element configured to modulate energy passing therethrough.In an embodiment, the third energy modulation elements and at least oneof the first and second energy modulation elements are located at thesame location, while in other embodiments, the first, second, and thirdenergy modulation elements are located at the same location or differentlocations.

An additional embodiment provides the ability to manufacturer the energywaveguide relay optics through a multi-step wafer level bonding process.Due to the resolution requirements of near field display and the highpixel densities required, and approach similar to other disclosuresdiscussed within this application for encoded energy waveguides forholographic super resolution may be additionally leveraged, with asophisticated modification to the energy waveguide relay design. Withhigh refresh rate synchronous energy modulation elements, two additionalhigher density energy waveguides are implemented at the center of eachof the two original energy waveguide pairs. The ratio of the increaseddensity may be the result of:D ² =Nn/Nc

where D is the increased density ratio needed (squared for X and Yrespectively), Nn is the new quantity of angular samples per waveguideelement desired, and Nc is the current system angular samples valuewithout the super resolution applied. For example, if the currentsampling provided 9 samples along X in the current system, and 27 aredesired, the density increase would be a factor of 3x².

In order to determine the number of time sequential samples that arerequired:FPSn=FPSs*D ²

where FPSn is the resulting required frame rate, FPSs is the nativecontent frame rate and D² is the ratio calculated from above. Forexample, if the source content frame rate is 24 fps and D² equals 9, thenew sampling frequency may comprise 216 fps.

FIG. 12 illustrates an orthogonal view of an alternative system 120 tothe waveguide relay system that directly inverts the rays of light suchthat one or more elements from system may be removed, in accordance withone embodiment of the present disclosure. This system 120 may implementa hardware modification wherein leveraging a reflective waveguide relaysystem 122A, 122B provides a direct inversion of every presentedpropagation path to a viewer's eye 124. This may further be advantageousfor the any of HMD systems or opacity generating devices such thatwaveguide relays may be removed from the overall system.

While various embodiments in accordance with the principles disclosedherein have been described above, it should be understood that they havebeen presented by way of example only, and are not limiting. Thus, thebreadth and scope of the invention(s) should not be limited by any ofthe above-described exemplary embodiments, but should be defined only inaccordance with the claims and their equivalents issuing from thisdisclosure. Furthermore, the above advantages and features are providedin described embodiments, but shall not limit the application of suchissued claims to processes and structures accomplishing any or all ofthe above advantages.

It will be understood that the principal features of this disclosure canbe employed in various embodiments without departing from the scope ofthe disclosure. Those skilled in the art will recognize, or be able toascertain using no more than routine experimentation, numerousequivalents to the specific procedures described herein. Suchequivalents are considered to be within the scope of this disclosure andare covered by the claims.

Additionally, the section headings herein are provided for consistencywith the suggestions under 37 CFR 1.77 or otherwise to provideorganizational cues. These headings shall not limit or characterize theinvention(s) set out in any claims that may issue from this disclosure.Specifically, and by way of example, although the headings refer to a“Field of Invention,” such claims should not be limited by the languageunder this heading to describe the so-called technical field. Further, adescription of technology in the “Background of the Invention” sectionis not to be construed as an admission that technology is prior art toany invention(s) in this disclosure. Neither is the “Summary” to beconsidered a characterization of the invention(s) set forth in issuedclaims. Furthermore, any reference in this disclosure to “invention” inthe singular should not be used to argue that there is only a singlepoint of novelty in this disclosure. Multiple inventions may be setforth according to the limitations of the multiple claims issuing fromthis disclosure, and such claims accordingly define the invention(s),and their equivalents, that are protected thereby. In all instances, thescope of such claims shall be considered on their own merits in light ofthis disclosure, but should not be constrained by the headings set forthherein.

The use of the word “a” or “an” when used in conjunction with the term“comprising” in the claims and/or the specification may mean “one,” butit is also consistent with the meaning of “one or more,” “at least one,”and “one or more than one.” The use of the term “or” in the claims isused to mean “and/or” unless explicitly indicated to refer toalternatives only or the alternatives are mutually exclusive, althoughthe disclosure supports a definition that refers to only alternativesand “and/or.” Throughout this application, the term “about” is used toindicate that a value includes the inherent variation of error for thedevice, the method being employed to determine the value, or thevariation that exists among the study subjects. In general, but subjectto the preceding discussion, a numerical value herein that is modifiedby a word of approximation such as “about” may vary from the statedvalue by at least ±1, 2, 3, 4, 5, 6, 7, 10, 12 or 15%.

As used in this specification and claim(s), the words “comprising” (andany form of comprising, such as “comprise” and “comprises”), “having”(and any form of having, such as “have” and “has”), “including” (and anyform of including, such as “includes” and “include”) or “containing”(and any form of containing, such as “contains” and “contain”) areinclusive or open-ended and do not exclude additional, unrecitedelements or method steps.

Words of comparison, measurement, and timing such as “at the time,”“equivalent,” “during,” “complete,” and the like should be understood tomean “substantially at the time,” “substantially equivalent,”“substantially during,” “substantially complete,” etc., where“substantially” means that such comparisons, measurements, and timingsare practicable to accomplish the implicitly or expressly stated desiredresult. Words relating to relative position of elements such as “near,”“proximate to,” and “adjacent to” shall mean sufficiently close to havea material effect upon the respective system element interactions. Otherwords of approximation similarly refer to a condition that when somodified is understood to not necessarily be absolute or perfect butwould be considered close enough to those of ordinary skill in the artto warrant designating the condition as being present. The extent towhich the description may vary will depend on how great a change can beinstituted and still have one of ordinary skilled in the art recognizethe modified feature as still having the required characteristics andcapabilities of the unmodified feature.

The term “or combinations thereof” as used herein refers to allpermutations and combinations of the listed items preceding the term.For example, A, B, C, or combinations thereof is intended to include atleast one of: A, B, C, AB, AC, BC, or ABC, and if order is important ina particular context, also BA, CA, CB, CBA, BCA, ACB, BAC, or CAB.Continuing with this example, expressly included are combinations thatcontain repeats of one or more item or term, such as BB, AAA, AB, BBC,AAABCCCC, CBBAAA, CABABB, and so forth. The skilled artisan willunderstand that typically there is no limit on the number of items orterms in any combination, unless otherwise apparent from the context.

All of the compositions and/or methods disclosed and claimed herein canbe made and executed without undue experimentation in light of thepresent disclosure. While the compositions and methods of thisdisclosure have been described in terms of preferred embodiments, itwill be apparent to those of skill in the art that variations may beapplied to the compositions and/or methods and in the steps or in thesequence of steps of the method described herein without departing fromthe concept, spirit and scope of the disclosure. All such similarsubstitutes and modifications apparent to those skilled in the art aredeemed to be within the spirit, scope and concept of the disclosure asdefined by the appended claims.

What is claimed is:
 1. A system comprising: a first energy waveguidesystem configured such that energy passing therethrough is directedaccording to a first 4D plenoptic function; a second energy waveguidesystem following the first energy waveguide system, the second energywaveguide system configured such that energy passing therethrough isdirected according to a second 4D plenoptic function, the second 4Dplenoptic function inverse of the first 4D plenoptic function; and afirst energy modulation element disposed in a first location in thefirst energy waveguide system, in a second location in the second energywaveguide system or in a third location in between the first energywaveguide system and the second energy waveguide system, the firstenergy modulation element configured to modulate a first energy passingtherethrough, wherein the first energy comprises energy entered throughthe first energy waveguide system from outside the first energywaveguide system and energy emitted withing the first energy waveguidesystem; wherein the first energy modulation element is operable toselectively modulate external energy received by the system from a firstexternal location on a first side of the system, and the first andsecond energy waveguide relay systems are operable to propagate, basedon the first and second 4D plenoptic functions, respectively, theexternal energy from the first external location on the first side ofthe system to a second external location on a second side of the system;and wherein the first energy modulation element is operable to generateenergy to be perceived, by a viewer at the second external location onthe second side of the system, as being superimposed with the externalenergy at the first external location on the first side of the system.2. The system of claim 1, wherein the first energy waveguide systemincludes a first array of energy waveguides configured to direct energytherethrough along a plurality of energy propagation paths, wherein theenergy waveguides of the first array are located at different spatialcoordinates, and each energy waveguide directs energy from therespective spatial coordinate to the plurality of energy propagationpaths along different directions according to the first 4D plenopticfunction.
 3. The system of claim 1, wherein the second energy waveguidesystem includes a second array of energy waveguides configured to directenergy therethrough along a plurality of energy propagation paths,wherein the energy waveguides of the second array are located atdifferent spatial coordinates, and each energy waveguide directs energyfrom the respective spatial coordinate to the plurality of energypropagation paths along different directions according to the second 4Dplenoptic function.
 4. The system of claim 1, further comprising asecond energy modulation element located in one of the first, second, orthird location, the second energy modulation element configured tomodulate energy passing therethrough.
 5. The system of claim 4, whereinthe first and second energy modulation elements are located at the samelocation.
 6. The system of claim 4, wherein the first and second energymodulation elements are located at different locations.
 7. The system ofclaim 4, further comprising a third energy modulation element located inone of the first, second, or third location, the third energy modulationelement configured to modulate energy passing therethrough.
 8. Thesystem of claim 7, wherein the third energy modulation elements and atleast one of the first and second energy modulation elements are locatedat the same location.
 9. The system of claim 7, wherein the first,second, and third energy modulation elements are located at the samelocation.
 10. The system of claim 7, wherein the first, second, andthird energy modulation elements are located at different locations. 11.The system of claim 7, wherein each of the first, second and thirdenergy modulation elements include LCD, LED, DLP, OLED, LCOS, andquantum dot.
 12. The system of claim 1, wherein at least one of thefirst energy waveguide system and the second energy waveguide system iscurved.
 13. The system of claim 1, wherein both the first energywaveguide system and the second energy waveguide system are curved. 14.A system comprising: a first energy waveguide system configured suchthat energy passing therethrough is directed according to a first 4Dplenoptic function; a second energy waveguide system following the firstenergy waveguide system, the second energy waveguide system configuredsuch that energy passing therethrough is directed according to a second4D plenoptic function, the second 4D plenoptic function inverse of thefirst 4D plenoptic function; a first energy modulation element disposedin a first location in the first energy waveguide system, in a secondlocation in the second energy waveguide system or in a third location inbetween the first energy waveguide system and the second energywaveguide system, the first energy modulation element configured tomodulate energy passing therethrough; and a second energy modulationelement located in one of the first, second, or third location, thesecond energy modulation element configured to modulate energy passingtherethrough; wherein the first energy modulation element is operable toselectively modulate external energy received by the system from a firstexternal location on a first side of the system, and the first andsecond energy waveguide relay systems are operable to propagate, basedon the first and second 4D plenoptic functions, respectively, theexternal energy from the first external location on the first side ofthe system to a second external location on a second side of the system;and wherein the first energy modulation element is operable to generateenergy to be perceived, by a viewer at the second external location onthe second side of the system, as being superimposed with the externalenergy at the first external location on the first side of the system.15. The system of claim 14, wherein the first energy waveguide systemincludes a first array of energy waveguides configured to direct energytherethrough along a plurality of energy propagation paths, wherein theenergy waveguides of the first array are located at different spatialcoordinates, and each energy waveguide directs energy from therespective spatial coordinate to the plurality of energy propagationpaths along different directions according to the first 4D plenopticfunction.
 16. The system of claim 14, wherein the second energywaveguide system includes a second array of energy waveguides configuredto direct energy therethrough along a plurality of energy propagationpaths, wherein the energy waveguides of the second array are located atdifferent spatial coordinates, and each energy waveguide directs energyfrom the respective spatial coordinate to the plurality of energypropagation paths along different directions according to the second 4Dplenoptic function.
 17. The system of claim 14, wherein the first andsecond energy modulation elements are located at the same location. 18.The system of claim 14, wherein the first and second energy modulationelements are located at different locations.
 19. The system of claim 14,further comprising a third energy modulation element located in one ofthe first, second, or third location, the third energy modulationelement configured to modulate energy passing therethrough.
 20. Thesystem of claim 19, wherein the third energy modulation elements and atleast one of the first and second energy modulation elements are locatedat the same location.
 21. The system of claim 19, wherein the first,second, and third energy modulation elements are located at the samelocation.
 22. The system of claim 19, wherein the first, second, andthird energy modulation elements are located at different locations. 23.The system of claim 14, wherein at least one of the first energywaveguide system and the second energy waveguide system is curved. 24.The system of claim 14, wherein both the first energy waveguide systemand the second energy waveguide system are curved.